Image processing apparatus

ABSTRACT

According to the present invention, an image processing apparatus is provided for inputting moving image data composed of a plurality of frames, for detecting definitions of a plurality of frames in the moving image data and a range of successive frames indicating an image similar to that of the image of the frame which is arbitrarily designated among the moving image data, and for selecting one of the frames in the moving image data which is inputted based on the definition and the similarity range.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention relates to an image processing apparatus,and more particularly, to a process for selecting a desired image froman inputted moving image.

[0003] 2. Related Background Art

[0004] Digital television receivers have been developed as apparatusesfor inputting and processing moving image data.

[0005]FIG. 24 shows the structure of such a digital television receiverin which, in particular, moving image data is received and a framedesignated by a user among received moving image data is outputted to aprinter, etc. (not shown) as a still image. Referring to FIG. 24,reference numeral 2401 denotes a broadcast receiving unit; 2402 denotesa received data analyzing unit; 2403 denotes a received data storingunit; 2404 denotes a moving image output unit; 2405 denotes a movingimage decoding unit; 2406 denotes a still image output unit; and 2407denotes an operation unit for designating an outputted frame.

[0006] Next, a description is given of an operation of the digitaltelevision receiver in FIG. 24 with reference to a flowchart of FIG. 25.In step S2501, the broadcast receiving unit 2401 receives digitaltelevision broadcast. In step S2502, the received data analyzing unit2402 analyzes received data, divides the analyzed data into moving imagedata, audio data, and other data, and stores the divided data into thereceived data storing unit 2403.

[0007] In step S2503, the moving image decoding unit 2405 decodesencoded moving image data, and outputs the decoded moving image data tothe moving image output unit 2406 on a frame basis. In step S2504, it ischecked whether or not the operation unit 2407 issues an instruction fordesignating an output of a still image. If YES in step S2504, theprocessing routine advances to step S2505. If NO in step S2504, theprocessing routine returns to step S2401. In step S2505, the designatedframe is outputted to a still image output unit 2408 as the still image.

[0008] As shown in FIG. 4, for example, a description is given toexplain an example in which in a television program 401, after a scene403 indicating that an anchorperson points out a flip chart, a scene 405indicating that the flip is closed up continues for several seconds and,then, is changed to a scene 407 indicating the face of the anchorperson.It is frequently desired that the scene 405 indicating theanchorperson's face closed-up is outputted to the printer as a stillimage so as to memorize the contents thereof.

[0009] In this case, in the aforementioned digital television receiver,a viewer operates the operation unit 2407, thereby outputting the frameat that moment (frame 401B shown in FIG. 4) to the printer to be printedas the still image.

[0010] However, although it maintains a sufficient picture quality aslong as it is viewed as the moving image, in many cases, a character,etc. become non-definitional if only a single frame is extracted as thestill image. If the same subject to be photographed is captured, thedefinition of the image is varied frame by frame because of blur and aout-of-focus state which is caused by slight movement of a camera. Ingeneral, the viewer designates the frame without recognizing theforegoing and, therefore, a non-definitional frame is often extractedand printed.

SUMMARY OF THE INVENTION

[0011] Accordingly, it is an object of the present invention to solvethe above-mentioned problems.

[0012] It is another object of the present invention to make it possibleto output a more definitional frame having the same contents as those ofa designated frame as a still image.

[0013] To accomplish the above-mentioned objects, according to an aspectof the present invention, there is provided an image processingapparatus comprising: input means for inputting a data string includingmoving image data composed of a plurality of frames; designating meansfor designating any desired frame among the moving image data;definition detecting means for detecting definitions of the plurality offrames in the moving image data; similarity range detecting means fordetecting from the moving image data a range of successive framesindicating an image similar to that of the frame designated by thedesignating means; and selecting means for selecting one frame in themoving image data which is inputted by the input means, based on anoutput of the definition detecting means and an output of the similarityrange detecting means.

[0014] Further objects, features and advantages of the present inventionwill become more apparent from the following description of thepreferred embodiments with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0015]FIG. 1 is a diagram showing one example of the structure of ahardware of a digital television receiver;

[0016]FIG. 2 is a diagram showing the structure of functions of an imageoutput device;

[0017]FIG. 3 is a flowchart for explaining a processing operation foroutputting a still image;

[0018]FIG. 4 is a diagram showing a specific example of a moving image;

[0019]FIG. 5 is a graph showing digitized degrees of similarity betweena designated frame and each frame;

[0020]FIG. 6 is a graph showing digitized degrees of similarity betweenany desired frame and a frame just before it;

[0021]FIG. 7 is a graph showing digitized definition of each frame;

[0022]FIG. 8 is a graph showing histogram of luminance of each pixel;

[0023]FIG. 9 is a graph showing spectrums of spatial frequencycomponents;

[0024]FIG. 10 is a diagram showing a motion vector for every block;

[0025]FIG. 11 is a diagram for explaining moving image data composed ofan I-picture, a P-picture, and a B-picture;

[0026]FIG. 12 is a diagram for explaining moving image data composed ofthe I-picture and P-picture;

[0027]FIG. 13 is a flowchart for explaining a processing operation foroutputting a still image;

[0028]FIG. 14 is a diagram showing a storing area of received data;

[0029]FIG. 15 is a diagram showing a storing portion of a scene changeflag;

[0030]FIG. 16 is a diagram showing a storing portion of definitioninformation;

[0031]FIG. 17 is a diagram showing the structure of functions of animage output apparatus;

[0032]FIG. 18 is a flowchart for explaining a processing operation foroutputting the still image;

[0033]FIG. 19 is a diagram showing a storing portion of output framedesignation data;

[0034]FIG. 20 is a flowchart for explaining a processing operation foroutputting the still image;

[0035]FIG. 21 is a diagram showing a status of the output framedesignation data and output candidate data;

[0036]FIG. 22 is a flowchart for explaining a processing operation foroutputting the still image;

[0037]FIG. 23 is a diagram showing a status of the output framedesignation data and scene change data;

[0038]FIG. 24 is a diagram showing the structure of functions of aconventional digital television receiver; and

[0039]FIG. 25 is a flowchart for explaining a processing operation forconventionally outputting the still image.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] Hereinbelow, a description will be given to explain embodimentsof an image output device, a method, and a computer-readable storagemedium according to the present invention.

[0041]FIG. 1 is a block diagram showing the structure of a hardware of adigital television receiver 100 according to a first embodiment of thepresent invention. Various antennas, a CATV terminal, a display, aprinter, an Internet line, etc. can be connected to the digitaltelevision receiver 100.

[0042] A viewer selects a channel by a remote controller 103, then, atuner 102 is controlled by a CPU 105, and broadcast data of a desiredchannel is obtained by the tuner 102 based on a signal inputted from theantenna or CATV terminal. According to the first embodiment, movingimage data, audio data, and other code data of a plurality of channelsare multiplexed as one stream, thereby obtaining broadcast data of onechannel in the digital television broadcast. A demultiplexer 104 dividesthe multiplexed data and stores the respective divided data in a memory108. The moving image data stored in the memory 108 is subjected todecoding and other processes by the CPU 105. The moving image data isoutputted to a display D via a display I/F 106.

[0043] Also, the moving image data, audio data, and other code data canbe distributed not only through the broadcast but also through theInternet via a modem 111. Alternatively, they can be distributed throughremovable media 110 such as a magnetic tape, magnetic disk, and opticaldisk. Or, data distributed through the broadcast, Internet, removablemedia 110 or the like is stored in an HDD 109 and the stored data isreproduced later. Although distribution through the broadcast isdescribed below, in place thereof, as needed, the distribution throughcommunication or a storage medium can be applied.

[0044] Further, from the respective frames forming the moving image, oneframe designated by the remote controller 103 can be outputted to aprinter P as a still image via a printer I/F 107. The respective blocksof FIG. 1 can mutually receive and transmit data therebetween through abus 112.

[0045]FIG. 2 is a block diagram showing the structure of functions inthe digital television receiver 100 shown in FIG. 1 in case of that, inparticular, the moving image data is received, a predetermined frame isselected upon designating the frame, and the frame is outputted to theprinter P as the still image.

[0046] Referring to FIG. 2, reference numeral 201 denotes a broadcastreceiving unit; 202 denotes a received data analyzing unit; 203 denotesa received data storing unit; 204 denotes a moving image output unit;205 denotes a moving image decoding unit; 206 denotes a similarity rangeobtaining unit; 207 denotes a definition obtaining unit; 208 denotes astill image output unit; 209 denotes a still image storing unit; 210denotes an output frame determination unit; and 211 denotes an operationunit. Incidentally, in the structure of the hardware shown in FIG. 1,the above components are formed by the tuner 102, remote controller 103,demultiplexer 104, CPU 105, memory 108, and the like.

[0047] Next, a processing operation for outputting the still image isdescribed with reference to a flowchart of FIG. 3.

[0048] In step S301, the broadcast receiving unit 201 receives digitaltelevision broadcast. In step S302, the received data analyzing unit 202analyzes the received data, divides the analyzed data into the movingimage data, audio data, and other data, and stores the divided data inthe received data storing unit 203. The received data storing unit 203has at least a storage capacity of an encoded data stream of duration ofseveral to several tens seconds. A data output timing from the receiveddata storing unit 203 to the moving image decoding unit 205 iscontrolled so as to hold the data stream of duration of several toseveral tens seconds preceding or subsequent to the frame outputted bythe moving image decoding unit 205, which will be described later.

[0049] In the first embodiment, the data stream to be received has beenencoded in accordance with an MPEG2transport stream (hereinafter,referred to as an MPEG2-TS) format using a DOT or motion compensationpredictive coding. FIG. 14 shows a data stream 1401 to be outputted bythe broadcast receiving unit 201 and a memory map 1402 indicating astatus of data to be stored in the received data storing unit 203.

[0050] In accordance with the MPEG2-TS, the encoded image, audio data,and other data are divided into a transport stream packet having 188bytes data (hereafter, referred to as a TS packet), and the TS packet ofthe image data, audio data, and other data are multiplexed at apredetermined timing.

[0051] The MPEG2 format uses the motion compensation predictive codingand variable-length coding, and the amount of codes thereof are varieddepending on contents of the image. In accordance with the MPEG2,intra-frame coding and inter-frame coding are combined to encode theimage data, and the amounts of codes are also different between theintra-frame coding (I-frame) and the inter-frame coding (P-frame andB-frame).

[0052] Consequently, the head of the one-frame image data encoded can behardly positioned at the head of the TS packet. There might exist aboundary portion between two frames in a single TS packet. In the firstembodiment, it is assumed to receive the data stream which has beenencoded so that the encoded data of 1 GOP (group of pictures) shown inFIG. 11, as a basic unit of MPEG2 coding, is distributed to an integernumber of TS packets. The GOP indicates a group ranging from the I-frameto a frame just before the next I-frame. Referring to FIG. 14, theencoded image data of the first one-GOP, and the audio data and otherdata in association therewith are composed of m TS-packets. The encodedimage data of the next one-GOP, and the audio data and other data inassociation therewith are composed of n TS-packets.

[0053] In step S303, the moving image decoding unit 205 decodes theencoded moving image data, and outputs the decoded moving image data tothe moving image output unit 204 on a frame basis. In step S304, it ischecked whether or not the operation unit 211 issues an instruction fordesignating an output of a still image. If YES in step S304, theprocessing routine advances to step S305. If NO in step S304, theprocessing routine returns to step S303.

[0054] In step 305, the output frame determination unit 210 searches thestart position of the frame similar to the designated frame (framedesignated by the operation unit 211) by tracing the past received data.In parallel therewith, the broadcast data is continuously received,stored, analyzed, and outputted in real time (in steps S301 and S303) tosearch as to how many frames similar to the designated frame arecontinued. The degree of similarity to the designated frame is obtainedby the similarity range obtaining unit 206 and a specific processthereof will be described hereinafter.

[0055] In step S306, the output frame determination unit 210 determinesthe most definitional frame between the start frame and the end framewithin the similarity range determined in step S305. The definitionobtaining unit 207 obtains the definition of the frame and a specificprocess thereof will be described later.

[0056] In step S307, the output frame determination unit 210 causes themoving image decoding unit 205 to decode the image data having the mostdefinitional frame determined in step S306 and outputting the decodeddata to the still image storing unit 209. The still image output unit208 reads the still image data stored in the still image storing unit209 at a proper timing and the read data is outputted to the printer P,etc.

[0057] According to the first embodiment, in the example in FIG. 4, whena viewer designates a frame 401B, a range of similarity is determined tobe an image ranging from a frame 401A to a frame 401D, which indicatethe similar image to that indicated by the designated frame 401B. In theimage, the most definitional frame 401C is outputted to the printer P.

[0058] Although the example for receiving the television broadcast isdescribed, in addition to the above-described example, when outputtingthe moving image which is distributed through the Internet or removablemedia, or when reproducing the moving image which is stored in the harddisk, the still image can also be outputted to the printer P in theabove manner.

[0059] In addition, the still image can not be only outputted to theprinter but also be displayed on a display screen as a temporary stopimage. Further, the still image can be distributed through the Internetand stored in the removable media or hard disk. Although the case wheredata is outputted to the printer P is described in the first embodimentand other embodiments, in place thereof, data can be outputted to ascreen, communication, or storage medium, as needed.

[0060] A detailed description is given of a processing operation in thesimilarity range obtaining unit 206, in other words, a obtaining methodof the degree of similarity of the frame.

[0061] The degree of similarity can be digitized by using a method forobtaining an absolute value of a difference between luminance data ofpixels corresponding to two frames and, then, adding up the absolutevalues of the differences of all pixels. In the case of the I-picture inthe encoded image data, of DCT coefficients of each of the blocks, a DCcomponent indicates an average luminance of each block. Therefore, thedegree of similarity can be obtained by using a method for obtaining theabsolute value of a difference between the DC components ofcorresponding blocks and, then, adding up the obtained absolute valuesof the differences of all blocks. In the case of the P- and B-pictures,the DC component of a reference frame is added and the resultant data isused as the DC component of the corresponding block, thereby detectingthe degree of similarity.

[0062] Note that, any of the above-mentioned methods can be used todigitize the degree of similarity and other methods can also be used.Further, a plurality of methods can be combined.

[0063]FIG. 5 shows a graph of the degree of similarity between thedesignated frame 401B and respective frames in the vicinity of the frame401B in FIG. 4. As shown in FIG. 5, the degree of similarity is obtainedwith respect to the designated frame 401B and the frame in the vicinitythereof, then, it is assumed that the frame is similar to the frame 401Bwhen the degree of similarity excesses a predetermined threshold value,and the frame is not similar if it does not exceed the predeterminedthreshold value. In FIG. 5, the range of the similarity is a frameportion having the degree of similarity of not greater than thepredetermined threshold value (before the frame 401A and after the frame401D). The similarity range obtaining unit 206 determines the range ofthe frames 401A to 401D as the similarity range, and outputs similarityrange information to the output frame determination unit 210.

[0064] A detailed description is given of a processing operation in thedefinition obtaining unit 207, that is, a method for obtaining thedefinition.

[0065] As shown in a graph 801 of FIG. 8, there are a plurality of peaksin obtained histogram of luminance of the respective pixels in theframes in the scene 405 indicating the closed-up flip in then example ofFIG. 4. If the blur of the camera or out-of-focus state is caused, theimage of the subject is formed with mixture of a high-luminance portionand a low-luminance portion. Thus, middle-luminance components areincreased as shown in a graph 802 of FIG. 8 and a standard deviation ofthe histogram is reduced. The determination of the standard deviationenables the definition to be digitized, and the definition is regardedto be higher when the standard deviation is larger.

[0066] Alternatively, if the blur or out-of-focus state of the camera iscaused in the image having a distribution of spatial frequencies asshown in a graph 902 of FIG. 9, low-frequency components are increasedin the spatial frequency. Therefore, when the image is encoded by theDCT etc., the definition can be digitized by obtaining the balancebetween the low-frequency component and the high-frequency component inthe spatial frequency of the image.

[0067] Alternatively, as shown in FIG. 10, it is possible to use motionvector information of every block upon encoding the image data, and theblur of the camera is reduced as the average of the motion vectors inone frame is smaller. Thus, the image is considered to be definitional.Therefore, the definition can be digitized by using the motion vector.

[0068] As shown in FIG. 11, among an I-picture (encoded frame withoutinter-frame prediction), a P-picture (frame determined by encoding adifference estimated from the predicted image based on the past frame),and a B-picture (frame determined by encoding a difference estimatedfrom the predicted image based on the past and future frames), ascomponents of the moving image, generally, the I-picture can have thebest picture-quality.

[0069] Hence, the above-described process may be executed by using theI-pictures, regardless of the P-picture and the B-picture. However,since the encoded data of the P-picture or B-picture immediatelybefore/after the subject frame includes a motion vector from theI-picture, when determining the motion vector of the I-picture, theseincluded motion vectors are used. The moving image composed of only theI- and P-pictures as shown in FIG. 12 also uses the above-describedmethod.

[0070] Although the above methods are used appropriately in the firstembodiment, the definition may be digitized by any of the foregoingmethods, by other methods, or by combining a plurality of methods.

[0071]FIG. 7 is a graph showing the definition of the respective framesin the example of FIG. 4. As shown in FIG. 7, in the similarity range ofthe frames 401A to 401D, the definition of the frame 401C is thehighest. The output frame determination unit 210 selects the frame 401Cbased on information on the similarity range transmitted from thesimilarity range obtaining unit 206 and information on the definitiontransmitted from the definition obtaining unit 207, and outputs theimage data of the frame 401C to the still image storing unit 209 throughthe moving image decoding unit 205.

[0072] If there are a plurality of frames having substantially identicaldefinition within the similarity range, those frames are assumed to havethe identical definition by disregarding a slight definition differenceand an output frame may be selected by taking into consideration otherconditions. For example, a frame having a higher degree of similarity tothe designated frame may be selected. Alternatively, a frame which isnearer to the designated frame with respect to time may be selected. Inthis case, strictly speaking, the “most definitional” frame is notoutputted. However, in the determination in step S306, it is assumedthat determination of the most definitional frame includes determiningthe frame having the definition which is substantially identical to thehighest one.

[0073] As described above, according to the first embodiment, the userdesignates the output of the still image, then, the range of thesuccessive frames having the image similar to the frame designated bythe user is detected, the most definitional frame is automaticallyselected within the detected range, and the image data of the selectedframe is outputted as the still image. Consequently, it is possible toindicate the image similar to that of the frame designated by the user,and to automatically select the more definitional frame.

[0074] Next, a description is given of a second embodiment of thepresent invention. The second embodiment employs the same structure ofthe hardware of the digital television receiver 100 in FIG. 1 and thesame structure of the functions of the image output device in FIG. 2.Hereinbelow, according to the second embodiment, a description is givento explain a processing operation for outputting the still image withreference to a flowchart in FIG. 13.

[0075] In step S1301, the output frame determination unit 210initializes a candidate of the output frame. In step S1302, the outputframe determination unit 210 turns off an internal frame designationoperation flag. In step S1303, the broadcast receiving unit 201 receivesthe digital television broadcast.

[0076] In step S1304, the received data analyzing unit 202 analyzes thereceived data, separates it into the moving image data, audio data, andother data, and stores them into the received data storing unit 203.

[0077] The received data analyzing unit 203 detects packets of the imagedata, audio data, and other data from the received MPEG2-TS, and storesthe detected packets in the received data storing unit 203 as shown in ascene 1402 in FIG. 14. Note that, in the structure of the secondembodiment, as will be described later, the received data storing unit203 may store only the data at least necessary for executing a decodingprocess by the moving image decoding unit 205 and does not have to storedata of duration of several to several tens seconds, as in the firstembodiment.

[0078] In step S1305, the moving image decoding unit 205 decodes theencoded moving image data, and outputs the decoded moving image data tothe moving image output unit 204 on a frame basis. In step S1306, theoutput frame determination unit 210 checks whether the internal framedesignation operation flag is ON or OFF. If the internal framedesignation operation flag is ON in step S1306, the processing routineadvances to step S1315. If OFF in step S1306, the processing routineproceeds to step S1307.

[0079] In step S1307, the similarity range obtaining unit 206 obtainsthe degree of similarity between a subject frame outputted to the movingimage output unit 204 from the moving image decoding unit 205 and aframe of a candidate of an output frame at present. The specific methodis the same as that of the first embodiment. In step S1308, it ischecked as to whether or not the degree of similarity detected is equalto a predetermined reference or more. If YES in step S1308, theprocessing routine advances to step S1310. If NO in step S1308, theprocessing routine proceeds to step S1309. In step S1309, a priorcandidate of the output frame is canceled, the frame decoded at presentis set to be a new candidate, and the routine proceeds to step S1310.

[0080] In step S1310, the definition obtaining unit 207 obtains thedegree of definition of the frame, as a subject, which is outputted tothe moving image output unit 204 from the moving image decoding unit205. The specific method thereof is the same as that of the firstembodiment.

[0081] In step S1311, the output frame determination unit 210 comparesthe degrees of definition between the subject frame outputted from themoving image decoding unit 205 with the frame which is the outputcandidate at present. If the subject frame outputted from the movingimage decoding unit 205 is more definitional than the frame which is theout put candidate, the processing routine advances to step S1313. On thecontrary, if not so in step S1311, the processing routine proceeds tostep S1312. Note that, when the prior candidate of the output frame iscanceled in step S1309, there is no object to be compared. Thus, thepresent frame becomes the candidate frame.

[0082] In step S1312, the output frame determination unit 210 stores theframe outputted by the moving image decoding unit 205 in the still imagestoring unit 209 as a new output image candidate and, in addition, italso stores the information on the degree of definition which isobtained in the definition obtaining unit 207. In step S1313, it ischecked whether or not there is an instruction for outputting the stillimage by the operation unit 211. If YES in step S1313, the processingroutine returns to step S1303. If NO in step S1313, the processingroutine advances to step S1314 whereupon the output frame determinationunit 210 turns on the internal frame designation operation flag. Thenthe processing routine returns to step S1303.

[0083] On the other hand, in step S1315, the similarity range obtainingunit 206 obtains the degree of similarity between the user's designatedframe and a frame subsequent thereto. The specific method thereof issimilar to that of the first embodiment.

[0084] In step S1316, the output frame determination unit 210 determineswhether or not a degree of similarity between the user's designatedframe and the subject frame outputted to the moving image output unit204 from the moving image decoding unit 205 is equal to a predeterminedreference or more, based on the information on the degree of similarityfrom the similarity range obtaining unit 206. If YES in step S1316, theprocessing routine advances to step S1310. If NO in step S1316, itproceeds to step S1317.

[0085] If the degree of similarity of both the frames are equal to thepredetermined reference or more in step S1316, that is, the framesimilar to the user's designated frame is outputted, the processingroutine proceeds to S1310 whereupon the output frame determination unit210 compares the degree of definition between the subject frame and thecandidate frame.

[0086] If the degree of similarity is less than the predeterminedreference in step S1316, the similarity range of the user's designatedframe includes the frames up to the frame just before the subject one.Thus, in step S1317, the output frame determination unit 210 outputs thelast one-frame image data remained as the output candidate of the stillimage, to the still image output unit 208 through the still imagestoring unit 209.

[0087] As mentioned above, according to the second embodiment, only theimage data of one frame is stored in the still image storing unit 209 asan output candidate of the still image. Therefore, the memory capacityof the received data storing unit 209 can be smaller than that of thefirst embodiment and costs can be further reduced.

[0088] Next, a description is given to explain a third embodiment of thepresent invention. The third embodiment employs the same structure ofthe hardware of the digital television receiver 100 in FIG. 1 and thesame structure of the functions of the image output device in FIG. 2.Hereinbelow, according to the third embodiment, a description is givenof a processing operation of the similarity range obtaining unit 206.

[0089] As shown in FIG. 5, the respective frames within the range of theframes 401A to 401D indicating the closed-up flip in the example of FIG.4 are exceedingly similar to the frame 401B. On the other hand, there isa low degree of similarity between the frame 401B and frames before theframe 401A and after the frame 401D.

[0090] According to the gist of the present invention, the similaritiesshould be directly compared between the frame designated by the viewerand the frames in the vicinity thereof. However, in the example of FIG.4, all the frames within the frames 401A to 401D can be assumed to havethe substantially same contents, and therefore, there is no problem evenif the most definitional frame is outputted as a still image from amongthe above frames, in place of the frame 401B. In other words, the scenechange portion is detected and it is determined whether or not thedetected portion belongs to the same scene. This determination can beregarded as indirect comparison of the similarity between the subjectframe and the viewer's designated frame.

[0091] As shown in FIG. 6, the scene change portion can include aportion with a extremely low degree of similarity to the frameimmediately before the scene change portion. The degree of similaritycan be digitized in the above manner described in the first embodiment.Alternatively, in the case of the moving image composed of the I-, P-,and B-pictures as shown in FIG. 11, or moving image composed of the I-and P-pictures as shown in FIG. 12, the difference is smaller when thedegree of similarity is higher. Therefore, a method for adding up thedifferences may be used.

[0092] Note that, since the means for detecting the scene change portionis available for other applications such as a scene search, variouskinds of equipment loading the means is expected to be presented in thefuture. In this case, this means can be applied to the third embodiment.

[0093] A description is given to explain a fourth embodiment of thepresent invention, in which a creator or editor of the moving image dataobtains the scene change in advance and information for designating theframe of the scene change is distributed as code data distributedtogether with the moving image data and the audio data.

[0094] In this case, on the transmission side, a data stream istransmitted by adding a flag indicating scene change to a header of a TSpacket including the encoded image data of the head of the frame as thescene change. On the reception side, the frame of the scene change canbe detected by detecting the information on the scene change from theheader of the TS packet in the image data of the received MPEG2-TS.

[0095] More specifically, in step S302 in the flowchart of FIG. 3, thereceived data analyzing unit 202 analyzes the headers of the TS packetsof the respective moving images in the received data stream, detects thescene change flag, and stores the detected scene change flag in otherdata storing areas of the received data storing unit 203 as shown inFIG. 15. In step S305, the similarity range obtaining unit 206determines the frame as the scene change based on the scene change flagshown in FIG. 15 and detects the similarity range.

[0096] As mentioned above, on the transmission side, a service foradding and distributing the flag showing the scene change is widely usedfor other applications such as the scene search. Therefore, this serviceis expected to be widely spread in the future. Since the digitaltelevision receiver does not need to perform the image processing todetect the scene by using the service, the calculation speed and thesize in program can be reduced and the present invention can be realizedwith the lower-cost configuration.

[0097] A description is given to explain a fifth embodiment of thepresent invention, in which a creator or editor of the moving image datadetermines the definition in advance and information on the definitionof the frame is distributed as code data distributed together with themoving image data and the audio data.

[0098] In this case, on the transmission side, a data stream istransmitted by adding information on the definition of the subject frameto a header of a TS packet including encoded image data of the head ofthe respective frames. On the reception side, the information ondefinition of the frame can be obtained by detecting the information ondefinition from the header of the TS packet in the image data of thereceived MPEG2-TS.

[0099] More specifically, in step S302 in the flowchart of FIG. 3, thereceived data analyzing unit 202 analyzes the headers of the TS packetsof the respective moving images in the received data stream, detects theinformation on the definition, and stores the detected information inother data storing areas of the received data storing unit 203 as shownin FIG. 16. In step S306, the definition obtaining unit 207 reads outthe information on the definition shown in FIG. 16 from the receiveddata storing unit 203 in accordance with an instruction from the outputframe determination unit 210, and outputs the read-out information tothe output frame determination unit 210. The output frame determinationunit 210 selects an image of one frame which is the most definitional,in accordance with the information on the definition.

[0100] As described above, if a service for distributing the informationon the definition is presented, since the digital television receiverdoes not need to perform the image processing to digitize the definitionby using the service, the calculation speed and the size in program canbe reduced and the present invention can be realized with the lower-costconfiguration.

[0101] Next, a description is given to explain a sixth embodiment of thepresent invention. The sixth embodiment employs the same structure ofthe hardware of the digital television receiver 100 in FIG. 1.

[0102]FIG. 17 is a block diagram showing the structure of functions ofan image output device for receiving, in particular, the moving imagedata and outputting a predetermined frame as a still image to theprinter, etc. when the frame is designated. Referring to FIG. 17,reference numeral 201 denotes a broadcast receiving unit; 202 denotes areceived data analyzing unit; 203 denotes a received data storing unit;204 denotes a moving image output unit; 205 denotes a moving imagedecoding unit; 208 denotes a still image output unit; 209 denotes astill image storing unit; 210 denotes an output frame determinationunit; and 211 denotes an operation unit.

[0103] In the sixth embodiment, differently from the first embodiment,the still image to be outputted is not designated by the similarityrange obtaining unit 206 or the definition obtaining unit 207, but it isdesignated by the output frame designation data included in the receiveddata stream. Note that, in the structure of the hardware shown in FIG.1, the above components are formed by the tuner 102, remote controller103, demultiplexer 104, CPU 105, memory 108, and the like.

[0104] Next, a processing operation for outputting the still image isdescribed with reference to a flowchart of FIG. 18.

[0105] In step S1801, the broadcast receiving unit 201 receives digitaltelevision broadcast. In step S1802, the received data analyzing unit202 analyzes the received data, divides the analyzed data into themoving image data, audio data, and other data, and stores the divideddata in the received data storing unit 203. In the sixth embodiment, thereceived data storing unit 203 also has at least a storage capacity of adata stream of duration of several to several tens seconds.

[0106] In the sixth embodiment, the received data stream includes anoutput frame designation data in the header of the TS packet containingthe head of the respective frames of the image data.

[0107] The output frame designation data is the data having contentssubstantially similar to the image of the subject frame and indicatingthe number of frames up to the more definitional frame. For example, ifthe output frame is the past one from the subject one, is the futureone, or coincides with the subject one, the number is a negativeinteger, a positive integer, or zero, respectively.

[0108] The received data analyzing unit 202 detects the framedesignation data from the header of the TS packet and, then, stores thedetected data in other data storage areas of the received data storingunit 203. In step S1803, the moving image decoding unit 205 decodes theencoded image data which is stored in the received data storing unit203, and outputs the decoded data to the moving image output unit 204 ona frame basis.

[0109] In step S1804, it is checked whether or not the operation unit211 instructs an output of the still image. If YES in step S1804, theprocessing routine advances to step S1805. If NO in step S1804, theprocessing routine returns to step S1803.

[0110] In step S1805, the output frame determination unit 210 obtainsthe output frame designation data for designating the output framecorresponding to the frame designated by the operation unit 211 fromamong the output frame designation data stored in the received datastoring unit 203. In parallel therewith, the output frame determinationunit 210 continues reception, storage, analysis, and output of thebroadcast data (in steps S1801 and S1803) in real time and, if theoutput frame is a frame after the designated one (corresponding to thefuture one), the output frame determination unit 210 waits untilreception of the output frame.

[0111] In step S1806, the output frame determination unit 210 reads outthe output frame from the received data storing unit 203, decodes theread-out data through the moving image decoding unit 205, and outputsthe decoded data to the still image storing unit 209. The still imageoutput unit 208 reads out one-frame image data which is stored in thestill image storing unit 209 at a proper timing, and outputs the imagedata to the printer P.

[0112] Referring to the example in FIG. 4, the frame designation dataindicating the number of frames up to the most definitional frame 401Camong the scene 405 is added to the TS packet header of each frame inthe scene 405 indicating the substantially same image, i.e., theenlarged image of the flip. If the frame 401C is a frame which is twentyframes after the frame 401B and the viewer designates the frame 401B,the output frame determination unit 210 recognizes that the frame to beoutputted is the one which is twenty frames after the frame 401B inaccordance with the frame designation data of the frame 401B. The outputframe determination unit 210 selects the frame 401C as the frame whichis twenty frames after the frame 401B, and stores the image data of theframe 401C in the still image storing unit 209.

[0113] As mentioned above, according to the sixth embodiment, byutilizing the frame designation data added on the transmission side, itis possible to easily detect the more definitional frame having thecontents similar to those of the user's designated frame.

[0114] Next, a description is given to explain a seventh embodiment ofthe present invention. The seventh embodiment employs the same structureof the hardware of the digital television receiver 100 in FIG. 1 and thesame structure of the functions of the image output device in FIG. 17.Hereinbelow, according to the seventh embodiment, a description is givenof a processing operation for outputting a still image with reference toa flowchart of FIG. 20.

[0115] In step S2001, the output frame determination unit 210initializes a candidate of an output frame. In step S2002, the outputframe determination unit 210 turns off an internal frame designationoperation flag. In step S2003, the broadcast receiving unit 201 receivesdigital television broadcast.

[0116] In step S2004, the received data analyzing unit 202 analyzes thereceived data, separates the analyzed data into the moving image data,audio data, and other data, and stores the separated data into thereceived data storing unit 203.

[0117] In the seventh embodiment, the received data stream includes theoutput frame designation data in a header of a TS packet containing ahead portion of each frame of the image data. Further, in the seventhembodiment, the frame designated as the output frame by other framesincludes an output candidate data in the header of the TS packetcontaining the head portion of the subject frame. The output candidatedata is not added to the frame which does not correspond to the outputcandidate by any other frames.

[0118] For example, in FIG. 4, when the frame 401C is designated by therespective frames in the scene 405 as the output frame, the outputcandidate data is added to the header of the TS packet including theframe 401C.

[0119] The received data analyzing unit 202 detects the framedesignation data and the output candidate data from the received datastream, and stores the detected data in other data storing areas of thereceived data storing unit 203 as shown in FIG. 21.

[0120] Note that, in the seventh embodiment, as will be described later,although the received data storing unit 203 may store only the necessaryand minimum data for the decoding process by the moving image decodingunit 205, it does not need to store data of duration of several toseveral tens seconds as mentioned above in the first embodiment.

[0121] In step S2005, the moving image decoding unit 205 decodes theencoded image data which is stored in the received data storing unit203, and outputs the decoded data to the moving image output unit 204 ona frame basis.

[0122] In step S2006, the output frame determination unit 210 checkswhether or not the output candidate data is added to the frame which isdecoded and outputted by the moving image decoding unit 205. If YES instep S2006, the processing routine proceeds to step S2007. If NO in stepS2006, the processing routine advances to step S2009.

[0123] In step S2007, the output frame determination unit 210 stores theimage data of the subject frame decoded by the moving image decodingunit 205 at present, as a new output image candidate, in the still imagestoring unit 209. In step S2008, the output frame determination unit 210checks whether or not an internal frame designation operation flag isturned on. If YES, the processing routine proceeds to step S2013. If NO,it proceeds to step S2009.

[0124] In step S2009, it is checked whether or not there is given aninstruction of the operation unit 211 during the decoding process of thesubject frame. If NO, the processing routine returns to step S2005. IfYES, it advances to step S2010.

[0125] In step S2010, the output frame determination unit 210 obtainsthe output frame designation data added to the user's designated framefrom the received data storing unit 203. As mentioned in the sixthembodiment, the output frame designation data can be indicated by thenumber of frames from the designated frame to the output one. If theoutput frame is a past one before the designated one, the number offrames is a negative integer. If the output frame is a future one afterthe designated one, the number of frames is a positive integer.

[0126] In step S2011, it is checked whether or not the output frame is afuture one. If YES, the processing routine proceeds to step S2012. IfNO, it advances to step S2013. In step S2012, the frame designationoperation flag is turned on and the processing routine returns to stepS2005. In step S2013, the output frame is not a future one, the imagedata of the output candidate of the still image has been already storedin the still image storing unit 209, and the output frame determinationunit 210 outputs to the still image output unit 208 the image data ofthe frame stored in the still image storing unit 209.

[0127] Note that, according to the seventh embodiment, the past framecorresponds to the one before the frames stored as the output candidateat the time of the frame designation operation, and the future framecorresponds to the one after the frames which is a new output candidateat the time of the frame designation operation. Under the condition thatthe past frame and the future frame do not become the output frame, theseventh embodiment is satisfied. However, the condition is sufficientlysatisfied on practical use and one frame suffices for the image storagecapacity. Consequently, the configuration of the seventh embodiment canbe realized with reduced costs.

[0128] Next, a description is given to explain an eighth embodiment ofthe present invention. The eighth embodiment employs the same structureof the hardware of the digital television receiver 100 in FIG. 1 and thesame structure of the functions of the image output device in FIG. 17.Hereinbelow, according to the eighth embodiment, a description is givenof a processing operation for outputting a still image with reference toa flowchart of FIG. 22.

[0129] In step S2201, the output frame determination unit 210initializes a candidate of an output frame. In step S2202, the outputframe determination unit 210 turns off an internal frame designationoperation flag. In step S2203, the broadcast receiving unit 201 receivesdigital television broadcast.

[0130] In step S2204, the received data analyzing unit 202 analyzes thereceived data, separates the analyzed data into moving image data, audiodata, and other data, and stores the separated data in the received datastoring unit 203.

[0131] In the eighth embodiment, the frame which is the output framedesignated by the other frames includes the output candidate data in aheader of a TS packet containing a head portion of the subject frame.Further, as mentioned in the fifth embodiment, on the transmission side,besides the output candidate data, a flag indicating the scene change isadded to the header of the TS packet including the encoded image data ofthe head portion of the scene change frame, and the data stream is thentransmitted.

[0132] The received data analyzing unit 202 detects the output candidatedata and the scene change data from the received data stream, and storesthe detected data in other data storing areas of the received datastoring unit 203 as shown in FIG. 23.

[0133] In step S2205, the moving image decoding unit 205 decodes themoving image data which is outputted from the received data storing unit203, and outputs the decoded data to the moving image output unit 204 ona frame basis. In step S2206, the output frame determination unit 210checks whether or not the scene change data as shown in FIG. 23 is addedto the frame which is outputted by the moving image decoding unit 205.If YES in step S2206, the processing routine proceeds to step S2007. IfNO in step S2206, the processing routine advances to step S2209.

[0134] In step S2207, the output frame determination unit 210 checkswhether or not an internal frame designation operation flag is turnedon. If YES, the processing routine proceeds to step S2206. If NO, itadvances to step S2208.

[0135] In step S2208, the output frame candidate is initialized. In stepS2209, the output frame determination unit 210 checks whether or not theoutput candidate data as shown in FIG. 23 is added to the frame which isoutputted by the moving image decoding unit 205. If YES, the processingroutine proceeds to step S2210. If NO, it proceeds to step S2212.

[0136] In step S2210, the image data of the frame added with thecandidate data is stored in the still image storing unit 209 as a newoutput image candidate. In step S2211, the output frame determinationunit 210 checks whether or not the internal frame designation operationflag is turned on. If YES, the processing routine proceeds to stepS2216. If NO, the processing routine advances to step S2212.

[0137] In step S2212, it is checked whether or not the operation unit211 issues an instruction for outputting the still image. If NO, theprocessing routine returns to step S2205. If YES, it advances to stepS2213.

[0138] In step S2213, the output frame determination unit 210 checkswhether or not the image data which is to be a candidate of the outputframe is stored in the still image storing unit 209. If YES, theprocessing routine proceeds to step S2216. If NO, that is, the data isstill in the initialized state and new data is not additionally stored,the processing routine advances to step S2214.

[0139] In step S2214, stored in the still image storing unit 209 is theimage data of the subject frame outputted by the moving image decodingunit 205 (that is, frame designated by the operation unit 211) as atemporary output frame candidate. This is the reason why that the framedesignated by the operation unit 211 is outputted as the still imagewhen there is no frame added with the output candidate data during aperiod from the frame added with the scene change data just before thedesignated frame to that just after the designated frame (i.e., periodin which it is assumed that the frames indicate the same scene).

[0140] In step S2215, the frame designation flag is turned on and theprocessing routine returns to step S2215. In step S2216, the still imageoutput unit 208 reads out the image data of the candidate frame of thestill image output which is stored in the still image storing unit 209,and outputs it.

[0141] Since the eighth embodiment employs the scene change data, it isunnecessary to add the data to designate the output frame for each frameon the transmission side. A service for distributing the scene changedata as the code data is available in a variety of cases, e.g., thescene search. Therefore, this service is expected to be widely spread inthe future. Since only the output candidate data may be added by usingthe service, the eighth embodiment can be implemented by modifying thestructure for distributing the moving image data to a small extent.

[0142] The objects of the present invention are accomplished bysupplying to a computer (CPU or MPU) in an apparatus or system connectedto various devices, program codes of software for implementing thefunctions of the first to eighth embodiments, and by operating thevarious devices in accordance with the program which is stored in thecomputer.

[0143] In this case, the program codes themselves of the softwareimplement the functions of the first to eighth embodiments. The presentinvention includes the program codes themselves and means for supplyingthem to the computer, for example, a storing medium for storing theprogram codes. The storing medium for storing the program codes may beany one of a floppy disk, hard disk, optical disk, magneto-optical disk,CD-ROM, magnetic tape, nonvolatile memory card, ROM, and the like.

[0144] Also, when the functions of the first to eighth embodiments canbe effected not only by executing the program codes supplied by thecomputer, but also by cooperation of the program codes and an OS(Operating System) or other application software operating on thecomputer, the above embodiments include the program codes.

[0145] Further, the present invention includes a case wherein thefunctions of the aforementioned respective embodiments can be effectedby storing the supplied program codes in a memory provided in a functionexpansion board inserted to the computer or a function expansion unitconnected to the computer, and by thereafter executing a part or all ofthe actual processes by the CPU, etc. provided for the functionexpansion board or function expansion unit on the basis of theinstruction of the program codes.

[0146] As mentioned above, according to the first to eighth embodimentsof the present invention, it is possible to output the more definitionalframe having the same contents as those of the designated frame as thestill image. Accordingly, it is possible to obtain and print out thestill image having the same contents in which characters, etc. aredefinitional.

[0147] While the present invention has been described with reference towhat is presently considered to be the preferred embodiments, it is tobe understood that the invention is not limited to the disclosedembodiments. On the contrary, the invention is intended to cover variousmodifications and equivalent arrangements included within the spirit andscope of the appended claims. The scope of the following claims is to beaccorded with the broadest interpretation so as to encompass all suchmodifications and equivalent structures and functions.

What is claimed is:
 1. An image processing apparatus comprising: inputmeans for inputting a data train including moving image data composed ofa plurality of frames; designating means for designating an arbitraryframe among said moving image data; definition detecting means fordetecting definitions of a plurality of frames in said moving imagedata; similarity range detecting means for detecting from among saidmoving image data a range of successive frames indicating an imagesimilar to that of the frame designated by said designating means; andselecting means for selecting one frame from the moving image datainputted by said input means based on an output of said definitiondetecting means and an output of said similarity range detecting means.2. An apparatus according to claim 1, wherein said selecting meansselects the most definitional frame from a plurality of frames withinsaid similarity range.
 3. An apparatus according to claim 1, whereinsaid definition detecting means detects the definition by using saidmoving image data.
 4. An apparatus according to claim 1, wherein saiddata train includes additional information indicating the definitions ofsaid plurality of frames, and said definition detecting means detectsthe definition based on the additional information in the data trainwhich is inputted by said input means.
 5. An apparatus according toclaim 1, wherein said similarity range detecting means detects thesimilarity range by using the moving image data.
 6. An apparatusaccording to claim 1, wherein the data train includes additionalinformation indicating scene change of said moving image data, and saidsimilarity range detecting means detects the similarity range based onthe additional information.
 7. An apparatus according to claim 1,wherein said similarity range detecting means compares the image data ofsaid designated frame with the image data of the plurality of frames insaid moving image data, and detects the similarity range based on thecomparison result.
 8. An apparatus according to claim 1, wherein saidsimilarity range detecting means compares image data of two adjacentframes in the moving image data, and detects the similarity range basedon the comparison result.
 9. An image processing apparatus comprising:input means for inputting a data train including moving image datacomposed of a plurality of frames; designating means for designating anarbitrary frame among said moving image data; similarity detecting meansfor detecting a similarity between the image data of two adjacent framesin said moving image data; definition detecting means for detectingdefinitions of a plurality of frames in said moving image data; andselecting means for selecting one frame from said moving image databased on an output of said similarity detecting means and an output ofsaid definition detecting means, and for storing the selected frame instoring means, wherein said selecting means reads out and outputs theimage data of one frame which is stored in said storing means inaccordance with a designating operation of said designating means. 10.An image processing apparatus comprising: input means for inputting adata train including moving image data, definition information, andscene change information, said moving image data being composed of aplurality of frames, said definition information indicating a definitionof each frame in said moving image data, and said scene changeinformation indicating scene change of said moving image data; andselecting means for selecting one frame in the moving image data basedon the definition information and the scene change information.
 11. Animage processing apparatus comprising: input means for inputting a datatrain including moving image data and frame designation information,said moving image data being composed of a plurality of frames, and saidframe designation information being added to said plurality of framesand indicating a predetermined frame in said plurality of frames;designating means for designating an arbitrary frame among the movingimage data; and selecting means for selecting said predetermined framefrom the moving image data based on the frame designation informationadded to the frame which is designated by said designating means.
 12. Anapparatus according to claim 11, wherein the data train includesselecting candidate information which is added to the predeterminedframe, and said selecting means further selects the predetermined framebased on the selecting candidate information.
 13. An image processingapparatus comprising: input means for inputting a data train includingmoving image data, scene change information, and selecting candidateinformation, said moving image data being composed of a plurality offrames, said scene change information indicating scene change of saidmoving image data, and said selecting candidate information being addedto a predetermined frame in said moving image data; designating meansfor designating an arbitrary frame among the moving image data; andselecting means for selecting the predetermined frame from the movingimage data based on a designating operation of said designating means,the scene change information, and the selecting candidate information.